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ABSTRACT 



The state-by-state comparisons of results on the 1998 
National Assessment of Educational Progress (NAEP) present the literacy 
community with some good news and a bit of a puzzle: scores have improved, 
but the results cannot provide answers as to why one grade or state is doing 
better or worse than in the past. State-by-state results show that Colorado, 
Connecticut, Kentucky, Maryland, South Carolina, Virginia, and Washington 
were among the states that made statistically significant gains from 1994 to 
1998. Many experts questioned suggestions that states with strong standards 
movements improved, noting that standards have not been in place long enough. 
Comparisons across states are also dangerous in that data cannot be used to 
suggest specific reading instruction strategies unless the populations from 
state to state are homogeneous and similar in terms of numerous socioeconomic 
factors. Poverty has more to do with test scores than anything else. Urban 
high poverty states tend to do less well. Montana, which scored right behind 
Connecticut, has excellent public schools but also a stable student 
population that speaks English and comes from families tuned into the 
majority system of education. Although results of the NAEP are heartening, 
Americans have much to do in their quest to be a nation of educated people. 
Contains a chart listing average scale score, grade 4, from the 1998 NAEP; 
and a comparison of state versus national average reading scores for public 
schools. (RS) 
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NAEP state-by-state: Cautious 
conclusions 



When the results from the 1998 National Assessment of Educational Progress (NAEP) 
were released back in February, the state-by-state comparisons were held back because, 
officials said, of a technical problem in their computation. In March, the state-by-state 
comparisons were released, and once again the literacy community was presented with 
some good news and a bit of a puzzle. 

In both the U.S. nationwide and state-by-state results, there has been improvement over 
the 1994 scores, the last time NAEP was administered. The NAEP test, which is known 
as "the nation's report card," measures such trends well. 

But NAEP is not a study or experiment-it does not isolate variables to determine cause 
and effect. It only measures how well or poorly American students did since the last test. 
Studies happen in a clean, controlled environment; NAEP is a measure of the murky, 
messy, multivariate world of day-to-day life. 

Because of this, NAEP will not tell us the reasons that one grade or state is doing better 
or worse than it used to. We want and need to know why Johnny can't read well or why 
Janey can, but NAEP doesn't answer those questions. It does, however, provide hints, 
clues, and patterns on which knowledgeable observers can base an educated guess. 

Then there are observers with little or no knowledge who allow themselves a great deal 
more latitude in drawing grand conclusions; the media, politicians, and many other 
self-appointed experts in education have sometimes attributed poor NAEP showings to 
one single factor-in California in 19.94, it was the state's literature-based language arts 
program without very much compelling evidence. 

Fortunately, the results were good in 1998. The nationwide results showed regained 
ground for the fourth and twelfth grades and significant advancement for the eighth (see 
the April/May 1999 issue of Reading Today). That pattern was reflected in the state-by- 
state results, which generally showed states regaining their 1992 levels in 1998 following 
the general decline experienced in 1994. 
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Professional development 
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^ Cathy Roller, IRA's director of research and policy, is among those who offered 

O comments on the state-by-state results. While cautious about drawing unwarranted 

conclusions, she does see some significant patterns in the data. 
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"Interpreting NAEP state-by-state reresults is a tricky business,' " Roller says. "What the 
results do tell us is that the states fall into roughly three groups that can be distinguished 
from one another: an average group, an above average group, and a below average group. 

"In addition, the state-by-state results tell us which states show a statistically significant 
improvement. From 1994 to 1998, Colorado, Connecticut, Kentucky, Maryland, South 
Carolina, Virginia, and Washington were among the states that made statistically 
significant gains. The question everyone would like to have the answer to is, 'Why did 
these states improve while others did not 

"It is tempting to suggest, as Secretary Riley did, that states with strong standards 
movements improved. However, many experts questioned this interpretation, saying that 
standards have not been in place long enough to affect scores and that demographic shifts 
may be as' much responsible as standards efforts and curriculum changes. 

"Another factor that has been hypothesized as affecting the scores is emphasis on 
professional development for teachers. For example, Connecticut has invested heavily in 
grant programs that support early intervention reading programs, such as Reading 
Recovery, and include a strong teacher development component, summer schools, 
Saturday academies, and teacher training centers. Maryland has enacted a law that 
requires all teachers to have had four courses in reading instruction. 

"However, for every hypothesis cited there are examples of states that have also tried the 
strategy and not improved NAEP scores. It may be that it is a complex mix of strategies 
that is producing results. 

"Another issue is the question of who is taking the tests. Recently released data suggest 
that there have been changes in the percentage of special education students exempted 
from taking the tests. For example, Connecticut, which ranked first in the state-by-state 
comparisons, also had a sizable increase in children exempted-10 percent, up from 6 
percent in 1994. Two other states that showed statistically significant gains, Kentucky 
and Louisiana, also increased their exemption rates. Kentucky's was 4 percent in 1994; 10 
percent in 1998, and Louisiana's was 6 percent in. 1994, 13 percent in 1998." 



Complex causes 

IRA President Carol M. Santa also urges caution, and warns that the state-by-state results 
can mislead policy makers into making counterproductive decisions. 

"There are a couple of things to think about when examining NAEP scores," 

Santa says. "NAEP is basically a test of reading comprehension, and the test data suggest 
a general increase in comprehension scores. Such increases must be interpreted 
cautiously. 
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"The fourth, eighth, and twelfth graders taking the test are too old to be influenced by the 
most recent legislative actions concerning reading. They are more likely influenced by 
shifts that took place five to 10 years ago. If we examine states that improved the most it 
is likely they set in place programs over the past few years that addressed literacy broadly 
and encouraged increased reading, more writing, and a focus on early intervention. 

"For example, I am not surprised that Connecticut came out on top and also showed the 
most improvement in NAEP scores. Over the last few years, Connecticut has made a 
concerted effort to improve reading proficiencies, particularly for children in the poorest 
urban districts, by providing extra support for early intervention programs. 

"Staff development in improving comprehension instruction has also become a priority in 
Connecticut This focus on comprehension instruction plus an emphasis on early 
intervention may account for some of the increase in NAEP scores. 

"1 am always troubled about state-by-state comparisons of educational data like NAEP. 
Such comparisons become an unfair horse race and discouraging to states scoring on the 
bottom. 

"Comparisons across states are also dangerous in that the data cannot be used to suggest 
specific reading instruction strategies unless one can assume that the populations from 
state to state are homogeneous and similar in terms of numerous socioeconomic factors 
such as income, employment, ethnicity classroom size, etc. 

"1 most worry about prescribed quick fixes for states scoring on the lower end of the 
distribution-tighter standards, skills-based instruction, more assessments, top-down 
driven curriculum, and more restrictions on teacher decisions. 

"NAEP scores actually reflect deeper issues in our country. Poverty has more to 
do with test scores than anything else. Urban high-poverty states tend do less well. I look 
at my own state, Montana We scored right behind Connecticut. I would like to think that 
our performance is due to our remarkable education system. Yes, we do have excellent 
public schools, but our students make it easy for us to look good. English is spoken by 
everyone, the population is stable, and most children come from families tuned into the 
majority system of education. 

"I am afraid that our high performance has little to do with standards or the type of 
reading programs offered in schools. Instead, it has to do with competent teachers and 
children who come to school ready to learn. Besides that, our long, dark winters are 
highly conducive to reading!" 

More work needed 

IRA President-elect Carmelita Kimber Williams notes that while there is quite a bit of 
work yet to do, the good news about gains in reading achievement across the board is 
heartening. 



"According to the reports from the states," Williams says, "children are improving and 
making some progress in reading achievement. 

"We still have problems that need to be dealt with, as evidenced by the fact that 
economics play a strong part in the quality of education that a child receives. Children 
born into minority families are still playing catch-up when it comes to educational 
achievement. 

"But progress is also being made here, and this is important because we don't want to 
leave any child uneducated as we move toward strong leadership for the United States 
and for the world. 

"The United States continues to be strong because we do, in fact, seek to educate all our 
children. And it can be a brighter future for them if we provide them with the best 
resources available for achieving reading excellence and successful learning experiences. 
We have much work to do as we continue in our quest to be a nation of educated people. 

Copyright 1999 International Reading Association 
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Average scale score, Grade 4 

1992 1994 1998 



Nation 


215 


212 


215+ 


States 








Alabama 


207 


208 


211 


Arizona 


209 


206 


207 


Arkansas 


211 


209 


209 


California! 


202 


197 


202 


Colorado 


217 


213 


222**++ 


Connecticut 


222 


222 


232**++ 


Delaware 


213 


206 


212++ 


Florida 


208 


205 


207 


Georgia 


212 


207 


210 . 


Hawaii 


203 


201 


200 


Iowa! 


225 


223 


223 


Kansas! 


— 


— 


222 


Kentucky 


213 


212 


218*++ 


Louisiana 


204 


197 


204++ 


Maine 


227 


228 


225 


Maryland 


211 


210 


215+ 


Massachusetts! 


226 


223 


225 


Michigan 


216 


— 


217 


Minnesota! 


221 


218 


222 


Mississippi 


199 


202 


204* 


Missouri 


220 


217 


216 


Montana! 


— 


222 


226 


Nevada 


— 


— 


208 


New Hampshire! 


228 


223 


226 


New Mexico 


211 


205 


206 


New York! 


215 


212 


216 


North Carolina 


212 


214 


217** 


Oklahoma 


220 


■ — 


220 


Oregon 


— 


— 


214 


Rhode Island 


217 


220 


218 


South Carolina 


210 


203 


210++ 


Tennessee 


212 


213 


212 


Texas 


213 


212 


217 


Utah 


220 


217 


215** 


Virginia 


221 


213 


218+ 


Washington 


— 


213 


217+ 


West Virginia 


216 


213 


216 


Wisconsin! 


224 


224 


224 


Wyoming 


223 


221 


219* 


Other Jurisdictions 








District of Columbia 


188 


179 


182** 


DDESS 


— 


— 


220 


DoDDS 


— 


218 


223++ 


Virgin Islands 


171 


— 


178* 
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*• Indicates that the average scale score in 1998 was significantly different from that in 1992 
using a multiple comparison procedure based on all jurisdictions that participated both 
years. * Indicates that the average scale score in 1998 was significandy different from that 
in 1992 if only one jurisdiction is being examined ++ Indicates that the average scale score 
in 1998 was significantly different from that in 1994 using a multiple comparison proce- 
dure based on all jurisdictions that participated both years. + Indicates that the average 
scale score in 1998 was significantly different from that in 1994 if only one jurisdiction or 
the nation is being examined. 

— Indicates jurisdiction did not participate, t Indicates jurisdiction did not meet one or 
more of the guidelines for school participation. DDESS: Department of Defense Domestic 
Dependent Elementary and Secondary Schools. DoDDS: Department of Defense 
Dependents Schools (Overseas). NOTE: National results are based on the national as- 
sessment sample, not on aggregated state assessment samples. Differences between states 
and jurisdictions may be partially explained by other factors not included in this table. 
SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAER), 1992, 1994, and 1998 Reading Assessments. 
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1998 Grade 4 comparison of state vs. national 
average reading scores for public schools 



Above the 

national 

average 


Colorado 

Connecticut 

DDESS 


DoDDS 

Iowa! 

Kansas! 


Maine Montana! Wisconsin! 

Massachusetts! New Hampshire! 
Minnesota! Oklahoma 


At or Kentucky 

around the Maryland 
national Michigan 
average 


Missouri Oregon 
New York! Rhode Island 
North Tennessee 

Carolina 


Texas 

Utah 

Virginia 


Washington 
West Virginia 
Wyoming 


Below the 

national 

average 


Alabama 

Arizona 

Arkansas. 


California! 
Delaware 
District of 
Columbia 


Florida 

Georgia 

Hawaii 


Louisiana 

Mississippi 

Nevada 


New Mexico 
South Carolina 
Virgin Islands 



t Indicates jurisdiction did not meet one or more of the guidelines for school participation. 
NOTE: Differences between states and jurisdictions may be partially explained by other 
factors not included in these tables. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1998 Reading Assessment. 
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